217 research outputs found
Automatic Registration of RGBD Scans via Salient Directions
We address the problem of wide-baseline registration of
RGB-D data, such as photo-textured laser scans without
any artificial targets or prediction on the relative motion.
Our approach allows to fully automatically register scans
taken in GPS-denied environments such as urban canyon,
industrial facilities or even indoors. We build upon image
features which are plenty, localized well and much more
discriminative than geometry features; however, they suffer
from viewpoint distortions and request for normalization.
We utilize the principle of salient directions present in
the geometry and propose to extract (several) directions
from the distribution of surface normals or other cues such
as observable symmetries. Compared to previous work we
pose no requirements on the scanned scene (like containing
large textured planes) and can handle arbitrary surface
shapes. Rendering the whole scene from these repeatable
directions using an orthographic camera generates textures
which are identical up to 2D similarity transformations.
This ambiguity is naturally handled by 2D features and allows
to find stable correspondences among scans. For geometric
pose estimation from tentative matches we propose a
fast and robust 2 point sample consensus scheme integrating
an early rejection phase. We evaluate our approach on
different challenging real world scenes
Real-World Normal Map Capture for Nearly Flat Reflective Surfaces
Although specular objects have gained interest in recent
years, virtually no approaches exist for markerless reconstruction
of reflective scenes in the wild. In this work, we
present a practical approach to capturing normal maps in
real-world scenes using video only. We focus on nearly planar
surfaces such as windows, facades from glass or metal,
or frames, screens and other indoor objects and show how
normal maps of these can be obtained without the use of an
artificial calibration object. Rather, we track the reflections
of real-world straight lines, while moving with a hand-held
or vehicle-mounted camera in front of the object. In contrast
to error-prone local edge tracking, we obtain the reflections
by a robust, global segmentation technique of an
ortho-rectified 3D video cube that also naturally allows efficient
user interaction. Then, at each point of the reflective
surface, the resulting 2D-curve to 3D-line correspondence
provides a novel quadratic constraint on the local surface
normal. This allows to globally solve for the shape by integrability
and smoothness constraints and easily supports
the usage of multiple lines. We demonstrate the technique
on several objects and facades
Motion-From-Blur: 3D Shape and Motion Estimation of Motion-Blurred Objects in Videos
We propose a method for jointly estimating the 3D motion, 3D shape, and
appearance of highly motion-blurred objects from a video. To this end, we model
the blurred appearance of a fast moving object in a generative fashion by
parametrizing its 3D position, rotation, velocity, acceleration, bounces,
shape, and texture over the duration of a predefined time window spanning
multiple frames. Using differentiable rendering, we are able to estimate all
parameters by minimizing the pixel-wise reprojection error to the input video
via backpropagating through a rendering pipeline that accounts for motion blur
by averaging the graphics output over short time intervals. For that purpose,
we also estimate the camera exposure gap time within the same optimization. To
account for abrupt motion changes like bounces, we model the motion trajectory
as a piece-wise polynomial, and we are able to estimate the specific time of
the bounce at sub-frame accuracy. Experiments on established benchmark datasets
demonstrate that our method outperforms previous methods for fast moving object
deblurring and 3D reconstruction.Comment: CVPR 2022 camera-read
Rolling Shutter Stereo
A huge fraction of cameras used nowadays is based on
CMOS sensors with a rolling shutter that exposes the image
line by line. For dynamic scenes/cameras this introduces
undesired effects like stretch, shear and wobble. It has been
shown earlier that rotational shake induced rolling shutter
effects in hand-held cell phone capture can be compensated
based on an estimate of the camera rotation. In contrast, we
analyse the case of significant camera motion, e.g. where
a bypassing streetlevel capture vehicle uses a rolling shutter
camera in a 3D reconstruction framework. The introduced
error is depth dependent and cannot be compensated
based on camera motion/rotation alone, invalidating also
rectification for stereo camera systems. On top, significant
lens distortion as often present in wide angle cameras intertwines
with rolling shutter effects as it changes the time
at which a certain 3D point is seen. We show that naive
3D reconstructions (assuming global shutter) will deliver
biased geometry already for very mild assumptions on vehicle
speed and resolution. We then develop rolling shutter
dense multiview stereo algorithms that solve for time of exposure
and depth at the same time, even in the presence of
lens distortion and perform an evaluation on ground truth
laser scan models as well as on real street-level data
Exploiting line metric reconstruction from non-central circular panoramas
In certain non-central imaging systems, straight lines are projected via a non-planar surface encapsulating the 4 degrees of freedom of the 3D line. Consequently the geometry of the 3D line can be recovered from a minimum of four image points. However, with classical non-central catadioptric systems there is not enough effective baseline for a practical implementation of the method. In this paper we propose a multi-camera system configuration resembling the circular panoramic model which results in a particular non-central projection allowing the stitching of a non-central panorama. From a single panorama we obtain well-conditioned 3D reconstruction of lines, which are specially interesting in texture-less scenarios. No previous information about the direction or arrangement of the lines in the scene is assumed. The proposed method is evaluated on both synthetic and real images
Removing Objects From Neural Radiance Fields
Neural Radiance Fields (NeRFs) are emerging as a ubiquitous scene representation that allows for novel view synthesis. Increasingly, NeRFs will be shareable with other people. Before sharing a NeRF, though, it might be desirable to remove personal information or unsightly objects. Such removal is not easily achieved with the current NeRF editing frameworks. We propose a framework to remove objects from a NeRF representation created from an RGBD sequence. Our NeRF inpainting method leverages recent work in 2D image inpainting and is guided by a userprovided mask. Our algorithm is underpinned by a confidence based view selection procedure. It chooses which of the individual 2D inpainted images to use in the creation of the NeRF, so that the resulting inpainted NeRF is 3D consistent. We show that our method for NeRF editing is effective for synthesizing plausible inpaintings in a multi-view coherent manner, outperforming competing methods. We validate our approach by proposing a new and still-challenging dataset for the task of NeRF inpainting
- …